Skip to content

Conversation

@ssolson
Copy link
Contributor

@ssolson ssolson commented May 19, 2025

This PR modernizes the package configuration by migrating from setup.py to pyproject.toml and adds a new optional dependency group for examples. It also includes various code improvements and CI workflow updates.

  • Migrated from setup.py to pyproject.toml for modern Python packaging
  • Removed requirements.txt in favor of pyproject.toml dependency management
  • Added new optional dependency groups for pip installs: E.g:
    • examples: Includes all dependencies needed for running examples
    • all: Includes all module dependencies
    • Individual module groups (wave, tidal, river, etc.)
  • Updated GitHub Actions workflows to use pip install -e ".[all, dev]" instead of pip install -e .

@ssolson ssolson changed the base branch from main to develop May 19, 2025 13:50
@ssolson ssolson self-assigned this May 19, 2025
@ssolson ssolson added the Clean Up Improve code consistency and readability label May 19, 2025
@ssolson ssolson linked an issue May 19, 2025 that may be closed by this pull request
@ssolson ssolson changed the title Modern Python Install Modernize Package Configuration May 28, 2025
@ssolson ssolson marked this pull request as ready for review May 31, 2025 20:12
@ssolson
Copy link
Contributor Author

ssolson commented May 31, 2025

@akeeste this is ready for review

Copy link
Contributor

@akeeste akeeste left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ssolson Overall the structure of this PR looks good, but I found many differences in the specific module dependencies. From what I see in each module, we should have the following module-specific dependencies:

wave = [
    "scikit-learn>=1.5.1",
    "statsmodels>=0.14.2",
    "netCDF4>=1.7.1.post1",
    "pytz",
    "NREL-rex>=0.2.63",
    "beautifulsoup4",
    "requests",
]

tidal = [
    "netCDF4>=1.7.1.post1",
    "requests",
]

river = [
    "netCDF4>=1.7.1.post1",
    "requests",
]

dolfyn = [
    "h5py>=3.11.0",
    "h5pyd>=0.18.0",
    "netCDF4>=1.7.1.post1",
    "cartopy",
]

power = [
    
]

loads = [
    "fatpack",
]

mooring = [
    
]

acoustics = [
    
]

qc = [
    "pecos>=0.3.0",
]

utils = [
    "pecos>=0.3.0",
]

I also suggest that we alter tidal.performance to only import dolfyn.VelBinner instead of the entire dolfyn module so that it will not require all of the other DOLfYN dependencies.

The main item missing here is bottleneck, but I can't find where that is being used.

pyproject.toml Outdated
Comment on lines 114 to 123
"mhkit[wave]",
"mhkit[tidal]",
"mhkit[river]",
"mhkit[dolfyn]",
"mhkit[power]",
"mhkit[loads]",
"mhkit[mooring]",
"mhkit[acoustics]",
"mhkit[qc]",
"mhkit[utils]",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simplify using "all" to cover all source code dependencies

Suggested change
"mhkit[wave]",
"mhkit[tidal]",
"mhkit[river]",
"mhkit[dolfyn]",
"mhkit[power]",
"mhkit[loads]",
"mhkit[mooring]",
"mhkit[acoustics]",
"mhkit[qc]",
"mhkit[utils]",
"mhkit[all]",

@ssolson
Copy link
Contributor Author

ssolson commented Jun 6, 2025

@ssolson Overall the structure of this PR looks good, but I found many differences in the specific module dependencies. From what I see in each module, we should have the following module-specific dependencies:

wave = [
    "scikit-learn>=1.5.1",
    "statsmodels>=0.14.2",
    "netCDF4>=1.7.1.post1",
    "pytz",
    "NREL-rex>=0.2.63",
    "beautifulsoup4",
    "requests",
]

tidal = [
    "netCDF4>=1.7.1.post1",
    "requests",
]

river = [
    "netCDF4>=1.7.1.post1",
    "requests",
]

dolfyn = [
    "h5py>=3.11.0",
    "h5pyd>=0.18.0",
    "netCDF4>=1.7.1.post1",
    "cartopy",
]

power = [
    
]

loads = [
    "fatpack",
]

mooring = [
    
]

acoustics = [
    
]

qc = [
    "pecos>=0.3.0",
]

utils = [
    "pecos>=0.3.0",
]

I also suggest that we alter tidal.performance to only import dolfyn.VelBinner instead of the entire dolfyn module so that it will not require all of the other DOLfYN dependencies.

The main item missing here is bottleneck, but I can't find where that is being used.

Adam great catch. Based on your review I decided to implement a test for each pip optional dependency where the tests for that module need to pass. Having some trouble with http requests that are delaying this from passing but I believe the lazy loading of modules is what we need to get this working. I don't expect the API to accept the data request until tomorrow at this point to get the tests passing.

@ssolson
Copy link
Contributor Author

ssolson commented Jun 9, 2025

Current Issue

The dolfyn module has become a critical dependency for several other modules (tidal, river, loads, mooring) through its coordinate transformation functionality. This creates a tight coupling where:

  1. Any module that imports dolfyn (directly or indirectly) requires cartopy
  2. I just added a lazy loading cartopy in dolfyn as a temporary fix that:
    • Only works because we know exactly where cartopy is used
    • Could break if new dependencies are added to dolfyn in the future
    • Doesn't address the fundamental architectural issue of tight coupling

Given the current architecture, the pip selective dependency installation approach may not be the best solution. We need to discuss if we want to move forward with this through a Longer-term refactor to the codebase by creating clear boundaries between modules by implement dependency injection for shared functionality

This would allow for true module independence and more flexible dependency management in the future.

Copy link
Contributor Author

@ssolson ssolson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akeeste with #404 pulled in and a couple of clean up items addressed I believe this is ready for another review.

I am currently using lazy loading of cartopy in the Dolfyn module due to the tight coupling listed above but I am open to other solutions.

@ssolson ssolson requested a review from akeeste July 2, 2025 15:43
@akeeste
Copy link
Contributor

akeeste commented Jul 7, 2025

dolfyn.adp.discharge.discharge() is the DOLfYN function that is used across MHKiT and should be moved to mhkit.utils in the future so that it can be accessed across MHKiT more easily, eliminating the tight coupling between DOLfYN and the other MHKiT modules

@akeeste
Copy link
Contributor

akeeste commented Jul 7, 2025

One more comment: it would be greatly preferred if users could just use pip install mhkit to get all dependencies using a typical installation method. @akeeste look into this during a final review of this PR

@akeeste
Copy link
Contributor

akeeste commented Jul 15, 2025

I dug through the pyproject.toml documentation but could not find a way to additionally allow pip install mhkit to function the same way as pip install mhkit[all]. Conda install can still be a simple method for users while the pip install allows for more advanced workflows. This is acceptable for now and we can change later on if required. We need to update the install instructions on our documentation.

The error message in mhkit/__init__.py for known modules however is never reached to inform the user how to install the optional dependencies. I will rework this before merging. I also think that utils should be a core dependency since its common functionality that many MHKiT modules use

Copy link
Contributor

@akeeste akeeste left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests passing and approved with the most recent changes

@akeeste akeeste merged commit 68fe393 into MHKiT-Software:develop Jul 15, 2025
54 checks passed
akeeste added a commit to akeeste/MHKiT-Python that referenced this pull request Sep 30, 2025
This PR modernizes the package configuration by migrating from
`setup.py` to `pyproject.toml` and adds a new optional dependency group
for examples. It also includes various code improvements and CI workflow
updates.

- Migrated from `setup.py` to `pyproject.toml` for modern Python
packaging
- Removed `requirements.txt` in favor of `pyproject.toml` dependency
management
- Added new optional dependency groups for pip installs: E.g:
  - `examples`: Includes all dependencies needed for running examples
  - `all`: Includes all module dependencies
  - Individual module groups (wave, tidal, river, etc.)
- Updated GitHub Actions workflows to use `pip install -e ".[all, dev]"`
instead of `pip install -e .`

---------

Co-authored-by: Andrew Simms <[email protected]>
Co-authored-by: akeeste <[email protected]>
@akeeste akeeste mentioned this pull request Sep 30, 2025
akeeste added a commit that referenced this pull request Sep 30, 2025
v1.0.0
# MHKiT v1.0.0
## New Features
* Sound Exposure Level by @jmcvey3 in #388
* Add discharge function to MHKiT by @jmcvey3 in #385

## Functionality enhancements
* Fix for corrupted Nortek files by @jmcvey3 in #372
* Update integral length scale function by @jmcvey3 in #376
* Fix ever-changing RDI RiverPro depth bin ranges by @jmcvey3 in #378
* Allow clean functions to handle _avg variables by @jmcvey3 in #377
* IEC TS 62600 updates by @akeeste in #382
* MLER explanation updates/corrections by @rgcoe in #393
* Improve Nortek2 index file creator functions by @jmcvey3 in #397
* Read Sentinel V specific data packets by @jmcvey3 in #396
* Short list of VMDAS updates by @jmcvey3 in #405
* Allow user to specify universal Kolmogorov constant for TKE dissipation rate function by @jmcvey3 in #406
* Nortek Dual Profile Dataset Rotation by @jmcvey3 in #414

## Source code improvements
* Lint Tidal by @ssolson in #386
* Lint river module by @ssolson in #389
* Lint hindcast by @ssolson in #398
* Modernize Package Configuration by @ssolson in #400
* Configure specific warnings by @ssolson in #401

## Bug fixes
* Avoid failing to scan very large files by @jmcvey3 in #371
* Acoustics SPL bugfix by @jmcvey3 in #379
* DOLfYN/RDI: Set  `fs` to NaN when typical calculation methods yield error (#408) by @simmsa in #409

## Testing and Continuous Integration Updates
* Fix Jupyter Notebook tests running Python 3.13 by @ssolson in #380
* CI Test Clean Up: Mock USGS, Acoustic Tolerances by @ssolson in #404
* Speed up tests with concurrency checks to prevent duplicate workflows on PRs from develop into main or from main into develop by @akeeste
* Define MPLBACKEND to decrease intermittent matplotlib errors in tests by @akeeste

## Documentation and Examples
* Add WEC-Sim power performance example  by @akeeste in #395
* Update dolfyn function docstrings and associated notebooks by @jmcvey3 in #412
* Update examples by @akeeste in #417
* Update installation instructions in README.md by @akeeste
* Adjust acoustics test tolerances by @akeeste in #420

**Full Changelog**: v0.9.0...v1.0.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Clean Up Improve code consistency and readability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update MHKiT pip install to install particular modules

3 participants